Metascheduling: A Scheduling Model for Metacomputing Systems

Metacomputing is the seamless application of geographically-separated distributed computing resources to user applications. We consider the scheduling of metaapplications; applications consisting of multiple components that may communicate and interact over the course of the application. Components may be schedulable computations, remote servers or databases, remote instruments, humans-in-the-loop, etc. We divide metaapplications into three categories — concurrent, parallel, and pipeline (below). Concurrent is the classic meta-application in which a set of components each running in a single site are executing concurrently and exchanging data. Parallel is a special case of concurrent in which a component is replicated and distributed across multiple sites. Pipeline applications consist of components connected in a chain-like fashion. The scheduling of meta-applications across multiple sites, called metascheduling, is a complex problem especially when the network capacity is assumed to vary between sites. The metascheduler is a decision maker that computes candidate schedules that produce reduced completion time using application and site resource information (bottom). Cost models are used by the metascheduler to evaluate the potential set of candidate schedules for the application. The meta-application completion time TCT is defined in terms of the computation and communication time for each component (given n components, m sites, where 1 ≤ (i ∧ j) ≤ n and 1 ≤ (k ∧ l) ≤ m, Ci is the ith component, Sk is the k th site, and shown for pipeline and two types of concurrent meta-applications): TCT [pipeline] = ∑ comp_time [Ci, Sk]+ comm_time [Ci, Cj, Sk, Sl] TCT [concurrent] = max {comp_time [Ci, Sk]+ comm_time [Ci, Cj, Sk, Sl]} TCT [concurrent, overlap within components] = max {comp_time [Ci, Sk], comm_time [Ci,Cj,Sk,Sl]} The scheduling problem is to determine an assignment of components to site resources that minimizes TCT. This scheduling problem is NP-complete and an exhaus..